Skip to content

feat: extend moe model check to multimodal ones and add block quantization parameters#605

Open
llcnt wants to merge 5 commits intomainfrom
feat/make_qwen3_5_kernel_tunable
Open

feat: extend moe model check to multimodal ones and add block quantization parameters#605
llcnt wants to merge 5 commits intomainfrom
feat/make_qwen3_5_kernel_tunable

Conversation

@llcnt
Copy link
Copy Markdown
Collaborator

@llcnt llcnt commented Mar 31, 2026

Description

Just extend the model check, and the logic inside the moe_kernel_tuner.py, so that one can tune multimodel MoE (eg. model that contains a vision tower and a text model nested, as qwen3.5-32b).
And make the block quant shape (ie the size of the block on which the scales are computed for re-normalization before quantization) usable from the user.

Related Issue

Fixes #(issue number)

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Refactor (no functional change)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Testing

  • I added or updated tests covering my changes
  • Existing tests pass locally (uv run pytest -m "cpu and not slow")

For full setup and testing instructions, see the Contributing Guide.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code, especially for agent-assisted changes
  • I updated the documentation where necessary

@codacy-production
Copy link
Copy Markdown

codacy-production bot commented Mar 31, 2026

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 11 complexity . 0 duplication

Metric Results
Complexity 11
Duplication 0

View in Codacy

TIP This summary will be updated as you push new changes. Give us feedback

@llcnt llcnt changed the title feat: extend moe model check to multimodal ones feat: extend moe model check to multimodal ones and add block quantization parameters Apr 1, 2026
@llcnt llcnt marked this pull request as ready for review April 1, 2026 12:10
@llcnt llcnt requested review from gsprochette and sharpenb April 1, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant